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ABSTRACT 



This document is a report of the proceedings of the fifth 
National Monitoring Conference. The contents include a brief background on 
monitoring and a synopsis of each of the conference sessions. The opening 
plenary covered the states' monitoring experiences in the first year of the 
new federal process following the reauthorization of the Individuals with 
Disabilities Education Act and a presentation by staff from the U.S. 
Department of Education, Office of Special Education Programs (OSEP) . The 
breakout sessions that featured presentations by state monitoring personnel 
are also summarized. State-level presentations at the conference portrayed 
state monitoring systems undergoing significant systemic change. Every state 
described plans to align their monitoring procedures with the new OSEP 
approach, with some adding their own unique versions of the continuous 
improvement elements. In each case, the change effort started with a 
broad-based stakeholder working group/committee that helped to design, plan, 
and implement the new system. The adoption or enhancement of a district 
self-assessment is another common element of the revisions. The document 
closes with a summary and discussion of next steps. (CR) 
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Special Education (NASDSE) is a cooperative agreement funded by 
the Office of Special Education Programs of the U. S. Department of 
Education. The project carries out a variety of activities that provide 
information needed for program improvement, and promote the 
utilization of research data and other information for improving 
outcomes for students with disabilities. The project also provides 
technical assistance and information on emerging issues, and 
convenes small work groups to gather expert input, obtain feedback, 
and develop conceptual frameworks related to critical topics in 
special education. 
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ABSTRACT 



This document is a report of the proceedings of the fifth National Monitoring 
Conference held in North Carolina in October 1999. The contents include a brief 
background on monitoring, and a synopsis of each of the conference sessions. The 
opening plenary covered the experiences of states monitored in the first year of the 
new federal process, and a presentation by staff from OSEP (U. S. Department of 
Education, Office of Special Education Programs). The breakout sessions that 
featured presentations by state monitoring personnel are also summarized. The 
document closes with a summary and discussion of next steps. 
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REPORT OF THE 1999 NATIONAL MONITORING CONFERENCE 

INTRODUCTION 



Purpose and Background 

The topic of monitoring has been a major focus of Project FORUM’s work for over seven 
years. The first FORUM document on this topic, entitled Analysis of State Compliance Monitoring 
Practices, included data collected for the second National Monitoring Conference held in 1992. In 
1994 and 1997, FORUM produced in-depth reports of the two subsequent National Monitoring 
Conferences and included data from a national survey of monitoring practices completed by the 
Federal/Regional Resource Center Network (FRRC). 1 In October 1999, the fifth National 
Monitoring Conference was held in North Carolina, and this document is a report of the 
proceedings of that meeting. 

Federal Monitoring of States 

Since 1975, the U.S. Department of Education, Office of Special Education Programs (OSEP) 
has monitored state education agencies (SEAs) to assess state compliance with the Individuals with 
Disabilities Education Act (IDEA). Originally termed the Program Administrative Review, 
monitoring involves a series of activities used by OSEP to determine the extent to which a state is 
in compliance with the federal special education requirements. The monitoring process is comprised 
of a wide variety of activities that include examination of documents and visits by OSEP teams to 
carry out onsite verification of implementation. 

The most recent OSEP report on the federal monitoring process covers the 1996-97 school year 
and is described in Section IV of the 2& h Annual Report to Congress } The following excerpts from 
that document describe the major features of the current monitoring process. 



OSEP uses research, dissemination, demonstration, systems change, and other technical assistance 
strategies to provide State and local educational agencies with tools to assist them in improving 
teaching and learning. 

OSEP also recognizes, however, the critical importance of its compliance monitoring 
responsibility and activities to ensure compliance with Congress’ mandates. OSEP places the 
highest priority on compliance with those IDEA requirements that have the strongest relationship 
with improved services and results for students with disabilities and their families. The Office 
tailors its monitoring and technical assistance activities in each State to maximize positive impact 
on educational services and results for students in that State. 



'These documents are entitled State Compliance Monitoring Practices: An Update, and Report of the 1997 
National Monitoring Conference and are available from NASDSE. 

2 This document can be accessed on the Department of Education website at the following internet address: 
www.ed.eov/offices/OSERS/OSEP/OSEP98AnlRpt/ 
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Over the past 4 years, OSEP has worked intensively to reorient and strengthen its monitoring 
system so that it will— in conjunction with research, innovation, and technical assistance efforts— 
support systemic reform that produces better results for students with disabilities and ensures 
compliance. To ensure a strong accountability system, OSEP has emphasized strong and diverse 
customer input in the monitoring process; effective methods for ensuring compliance with Part 
B, with strongest emphasis on requirements that relate most directly to continuous improvement 
in learner results; prompt identification and correction of deficiencies; and corrective action 
requirements and strategies that yield improved access and results for students. 

The current approach used by OSEP is described in detail in the publication entitled Continuous 
Improvement Monitoring Process : 1999-2000 Monitoring Manual 3 The seven phases of the 
process are as follows: 

1. Self-Assessment 

2. Validation Planning 

3. Validation Data Collection 

4. Reporting to the Public 

5. Improvement Planning 

6. Implementation of Improvement Strategies 

7. Verification and Consequences 

The manual details each phase and includes appendices containing added guidance for states 
in meeting the requirements of the federal monitoring procedures. 

State Monitoring of LEAs 

States receive federal funds under the IDEA to support the education of students with 
disabilities. Since its original passage in 1975, this federal law has held SEAs responsible for 
monitoring how local education agencies (LEAs) provide programs and services for those students. 
The following portion of the IDEA as amended in 1997 is the current legislative wording of the 
state supervisory role: 

(11) STATE EDUCATIONAL AGENCY RESPONSIBLE FOR GENERAL SUPERVISION- 
(A) IN GENERAL- The State educational agency is responsible for ensuring that — 

(i) the requirements of this part are met; and 

(ii) all educational programs for children with disabilities in the State, including all such 
programs administered by any other State or local agency— 

(I) are under the general supervision of individuals in the State who are responsible 
for educational programs for children with disabilities; and 

(II) meet the educational standards of the State educational agency. [20 U.S.C. 

Chapter 33, Sec. 1412(a)] 



3 



This manual is available online at: www.dssdc.org/frc/monitor.htm 
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DESIGN AND OVERVIEW OF THE CONFERENCE 



The 1999 conference title, The A, B, C, Ds of Monitoring in the Next Millennium, was chosen 
to capture the importance of monitoring issues related to the entire spectrum of IDEA, specifically 
Parts B, C, D, and Section 619. The expressed purpose of the conference was, “To address state 
agency challenges in monitoring state and local programs under IDEA.” As with all the prior 
monitoring conferences, the major focus was state monitoring of LEAs, and the primary goal was 
the sharing of information among state monitoring personnel. OSEP personnel also participated 
and provided updates on federal issues. 

The conference was planned by the Monitoring Workgroup that included representatives from 
the Federal Resource Center (FRC), the Regional Resource Centers (RRCs), the National 
Association of State Directors of Special Education (NASDSE), The National Early Childhood 
Technical Assistance Project (NECTAS), and OSEP. Proposals were solicited from every SEA for 
presentations to be made at the conference. In addition to formal sessions, SEAs could offer to 
participate in the “Marketplace of IDEAS,” a sharing activity at which states could display their 
materials at a “poster” type session and inform others about the policies and procedures used in 
their state’s monitoring program. Conference participants represented 48 states and 4 non-state 
jurisdictions. The agenda sessions and the SEAs involved in the presentations were described 
below: 

Opening Plenary: Results-Based Monitoring of IDEA: Our Experience in Defining the Process 
(MPRRC, NECTAS, KS and UT) 

OSEP Plenary Session 
Breakout Sessions: 

Al. Monitoring Issues for Part C (NECTAS) 

A2. Statewide Electronic Data Management for Children with Disabilities: An Aid to 
Monitoring (MT and WY) 

A3. Focus on Success (LA) 

Bl. The Quality Part of Quality Assurance: Impacting on Program Design and Student 
Outcomes (NY) 

B2. The Quality Improvement Process: Collaboration Toward Quality (NE and MN) 

B3. Draft Monitoring Process for Compliance and Continuing Improvement (PA and WV) 
B3. The West Virginia Continuous Monitoring Process (WV) 

Cl. Special Education Monitoring: What’s New (TX) 

C2. Aligning State Monitoring Systems with OSEP Monitoring Model (IL, IN, WI, and MO) 
C3. Integrating Reform Efforts and Programs: Focused Monitoring (CA) 

Dl. Continuous Learning and Improvement Process: Part C - IDEA (VT) 

D2. Show-Me Student Outcomes: Monitoring for Results (MO) 

D3. Monitoring Through Technology: Self Assessment and Discipline Reporting (UT) 
Closing Plenary: State Innovative Practices (NM and KY) 

Marketplace of IDEAS: CT, DE, FL, GA, KY, NY, NC, TX, VT, and WI. 
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SESSION SUMMARIES 

Opening Plenary: Results-Based Monitoring of IDEA: Our Experience in Defining the Process 
Presenters: John Copenhaver. Director, MPRRC 
Wayne Ball. Specialist, MPRRC 
Tal Black. NECTAS 
Tracy Wohl. Planning Analyst, KS 
Donna Suter. Specialist, UT 

Eight of the 12 states monitored in 1998-99, the first year under OSEP’s new system, were in 
the MPRRC area. John Copenhaver reviewed the lessons learned by the those states in terms 
of six themes and the implications for SEA monitoring of LEAs. 

1. Past approaches resulted in a top-heavy emphasis on compliance issues; the current 
process still addresses compliance, but will also focus on those areas that have a direct 
relationship to results for children with disabilities. 

For SEAs: A similar shift in focus in state monitoring systems will require 
retraining of SEA monitoring staff and making LEAs aware of change. It 
was suggested that the performance goals and indicators that IDEA now 
requires states to set is one area in which early childhood programs and 
LEAs could begin to collect baseline data. 

2. Part C/Part B - Special education is increasingly being seen as a seamless system with 
transition from one process or level to another without barriers or interruptions in 
services; 

For SEAs: State monitoring systems should encourage collaboration 
between Part C, 6 1 9, and Part B including communication with community 
providers through their involvement in local steering committees. 

3. The new monitoring process includes increased stakeholder involvement; 

For SEAs: States should encourage LEAs to assemble their own local 
steering committees to conduct a self-assessment. 

4. Another new component — the optional self-assessment — allows states to better 
identify their own strengths and deficiencies; 

For SEAs: By empowering community members to conduct a self 
assessment and develop their own improvement plan, the odds will increase 
that change will occur to improve results for students with disabilities. 

5. The checks and balances in the methodology of the new process help to ensure 
reliability and validity of the data and findings through three main validation 
methods — self-assessment, public input, and on-site visits; 

For SEAs: A monitoring system needs to have a means of providing 
enough information to validate the self-assessment. 

6. A unique feature of the new process is that it is continuous, not a product or an event. 

For SEAs: The design of a SEA monitoring process can engage LEAs at 
various stages of improvement. Districts doing an excellent job in certain 
areas could be linked with those that need guidance. 
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Wayne Ball reviewed the outcomes of a Monitoring Summit meeting that the MPRRC 
convened for members of their region. Participants explored ways to improve collaboration 
among all concerned with monitoring — OSEP, NECTAS, Bureau of Indian Affairs, Part B 
monitoring personnel, 619 personnel, and Part C personnel. Some of the individuals from the 
same state were meeting each other for the first time. Significant progress was described as a 
result of this event. Current monitoring practices were reviewed and the elements of the new 
process were discussed. 

Tal Black addressed the issues of collaboration between Parts B and C. Some of the reasons 
that Parts B and C have gone their separate ways in monitoring include the fact that, in all but 
a dozen states, the lead agency for the Part C program is not the state education agency, making 
collaboration between Parts B and C on local monitoring an inter- agency, rather than an intra- 
agency, endeavor. In addition, Part C’s early intervention services are provided through a 
variety of local public and private agencies and rarely through the public schools. Eligibility 
for the two programs is also different, as is the level of support and involvement of the child’s 
family. 

Building a custom house provides a metaphor for how Parts C and B can relate to each other 
in supporting an individual with disabilities during her or his formative years. When you build 
a custom designed house many different craftsmen must be involved. Some aspects of the 
construction must precede others, but there is an overall plan that all seek to follow, a plan 
developed with and approved by the owner. Each craftsman can take pride in his or her work, 
and together they can share in the satisfaction of seeing a house being well built. But everyone 
involved understands from the beginning that the house belongs to the family. Monitoring is 
like the building inspectors that come periodically to see that the builders are following the 
prescribed building codes, using approved materials, are properly licensed. Local monitoring 
is also like having a consultant come to advise the builders and the family on sound design and 
construction techniques, to share new innovations from other building sites, and take away new 
ideas for others. Monitoring also helps assure that each house is custom designed to the unique 
needs of the child and family, not a “cookie cutter” house. 

Some specific areas for Parts C and B collaboration include the transition of children at age 
three, the use of “developmental delay” as an eligibility category, and the sharing of information 
to enhance the effectiveness of both types of programs. Both Parts C and B belong to one 
whole — the IDEA. Monitoring all parts should be a coordinated venture. 

Tracy Wohh from the KS Dept, of Health and Environment that is the lead agency for Part C, 
described the Kansas process for monitoring of 37 local early intervention networks that are part 
of a decentralized system with local decision making authority. The words “monitoring” and 
“compliance” are not used at all in the local self-assessment process. The entire process is 
carried out through a contract with the KS University Affiliated Program (KUAP). The KUAP 
helps in the development of self-assessments and in the onsite phase by providing all the 
recruitment and training of site visitors who include parents and family members, English 
language learners, agency, educational, business and university personnel. The KUAP also 
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helps with the actual visit including follow-up work and the provision of aggregated data for 
analysis. Technical assistance is provided during the years there is no site visit. This process has 
been used for six years and continues with the addition this year of a Planning Committee to 
explore improved collaboration between Parts B and C. 

Donna Suter described Utah’s revised monitoring, called the Utah Program Improvement 
Planning System, that is based on the model developed by OSEP. The objectives of the new 
system are to move to a continuous process focused on student results, to tie overall district and 
school improvement efforts to compliance with IDEA, and to improve the working relationship 
among stakeholders at all levels. 1999 is the pilot year for this new system that also includes 
efforts to highlight the strengths of programs and the provision of technical assistance where 
needed. The new system is a five-year cycle moving from self assessment, through four years 
of district-planned improvement plans and finally, ongoing evaluation. 



OSEP Plenary Session 

Presenters: OSEP Staff, Monitoring and Self Improvement Program: 

Gregg Corr 
Sheila Friedman 
Larry Ringer 
Carolyn Smith 
Sarah Willis 

Gregg Corr : Most of the 1997 amendments to IDEA went into effect immediately upon 
enactment of the Act on June 4, 1 997. 

Like many States, OSEP recognized the need to move quickly to better align its monitoring 
system with the 1 997 amendments to IDEA’S strong focus on improving results for children and 
youth with disabilities. As a result, OSEP has been “sailing its new monitoring process while 
it has been building it.” 

Many States have also been working on aligning their monitoring with OSEP, but there will be 
feedback and adoption both ways. OSEP appreciates the cooperation of states that piloted the 
new OSEP process. 

OSEP wants to preserve the strong points of the law and the old monitoring system, while 
expanding monitoring for results. Why shift to a results-based system? The old system that 
looked at procedures was an important system for its time. In the past, students were excluded 
from public education, and it was important to see that all the protections of the law were 
followed to correct that problem. But, through the 1990s, it became clear that an emphasis on 
procedural compliance did hot necessarily produce system change and improved results. The 
former system did not promote state ownership of the monitoring process. Monitoring was 
something OSEP did to the state as a discrete activity repeated periodically. Public awareness 
was not well integrated with the process. Public input often identified issues of an individual 
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nature and not information that could be used at the state level. Also, there were inadequate 
consequences even when the same type of IDEA violations were seen in subsequent monitoring. 
Until a year ago, there was no monitoring of Part C except for some sporadic isolated visits with 
no linkage to the Part B monitoring. So, the new system is designed to cover all of IDEA from 
birth to the end of the mandatory coverage period in an integrated way. 

In February 1 998, OSEP convened a stakeholders meeting that included SEAs, Lead Agencies, 
State Advisory Councils, Coordinating Councils, and regular and special educators who were 
asked to define a vision and strategies for a revised monitoring system to incorporate the themes 
for a new approach. The seven critical themes for the proposed new system were: 

❖ Continuity, rather than an episodic system, that integrated self-assessment with 
continuous feedback and response. 

❖ A real partnership with stakeholders. 

❖ State accountability'and active participation. 

❖ Self-assessment that is focused on improving results for students with disabilities. 

❖ A data-driven process, with each state collecting and using data aligned with its 

performance goals and indicators that is reviewed regularly by OSEP. 

❖ A public process with broad dissemination of all information. 

❖ Technical assistance including close working with the RRCs and NECTAS and the 

identification and dissemination of promising practices. 

Larry Ringer : IDEA has always been, and continues to be, a civil rights law designed to protect 
rights of children with disabilities and their families. As Congress has revised the statute to 
increase the focus on improving results for children with disabilities, it has retained these 
protections for the rights of each child. The critical challenge is to find the most effective 
balance between procedural compliance and improved results. 

There is real variance among the States in both the approach and the success of their efforts to 
ensure compliance. Some States have very effective systems for identifying and correcting 
noncompliance. Others find noncompliance, but cannot ensure that school districts and early 
intervention providers correct the problems. Still others are yet working to develop an effective 
system for identifying noncompliance. 

OSEP strongly supports the efforts of most States to implement monitoring systems with a 
strong emphasis on improving results for infants, toddlers, children and youth with disabilities. 
Such a focus cannot, however, result in a weakening of a State’s effectiveness at identifying and 
correcting noncompliance. 

Monitoring needs will vary from State to State, depending upon a number of factors, including 
the historical effectiveness of the State’s systems for general supervision and the extent to 
which districts have complied with Part B requirements. In States where there is a lot of long- 
standing serious noncompliance, it is especially important that the State focus efforts on 
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achieving compliance. The steering committee(s) in each State will be critical in helping to 
identify monitoring needs. 

Compliance is not only a set of paperwork “technicalities” — it includes major issues such as 
least restrictive environment (LRE), and free appropriate public education (FAPE). Each state 
has to build a system that will work for it. The trust level is an important issue for families, 
LEAs, SEAs, Lead Agencies, and everyone involved. While the most critical thing for families 
is results, it is still critical that the IEP process and the IEP document be accurate and 
appropriate for their child. 

It is important that SEAs be able to aggregate and disaggregate data, (e.g., by type of disability), 
to ensure that the system is addressing the needs of all children. This is really a part of general 
supervision — not only attention to “dotted i’s and crossed t’s,” but seeing how all the parts of 
the law are being implemented. It will take some time to build the necessary databases, and it 
is clear that progress in this area varies among states. SEA monitoring needs to be tailored to 
each LEA in the same way that OSEP must tailor its process for each state. 

Consequences for non-compliance: The development of corrective actions that will ensure 
effective correction and improved results is a challenging task. The critical issue is to identify 
corrective actions that will result in documented change in practices and results. 

Both OSEP and many States are working to identify a number of positive and negative 
consequences that will help ensure prompt and effective corrective action. 

OSEP’s continuous improvement monitoring process can best be conceived of as a continuous 
circle with seven parts: 

1 . Self-assessment - in some cases, Parts B and C are combined. 

2. Validation Planning - OSEP comes into the state to plan, analyze the self- 

assessment, hold public focus meetings, meet with the steering committee, and 
develop a strategy for OSEP’s second visit to the state. 

3. Visit (validation data collecting) - the OSEP team travels to the state to see 

programs, talk with parents and others, and provide an initial report of findings 
on the last day of that visit. 

4. Report to the Public - includes compliance areas, strengths of the system, public 

input, and areas that are not compliance violation, but that could bring 
improvements. 

5. Improvement Planning - OSEP works with state to decide what needs to be done 

and how it will be demonstrated that system change has occurred. 

6. Implementation of Improvement Strategies - the state implements its improvement 

plan and evaluates its effectiveness. 

7. Verification and Consequences - including enforcement actions that go beyond 

typical corrective actions. 
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Many challenges still lie ahead under this new system. It is clear at this point that results-based 
monitoring systems are much more challenging and demanding for everyone involved. An 
ongoing challenge is how to focus on results when the federal standards are largely procedural. 
OSEP took each requirement and discussed the intended impact of that requirement on 
improved results. Then, OSEP developed five Part C and four Part B cluster areas to focus on 
both improved results and compliance that supports such results. 

Some requirements that have the greatest impact on children are the most difficult to monitor. 
For example, it is easy to check whether a parent has given consent, but much more difficult 
to monitor the delivery of services. The 1997 amendments to IDEA emphasizes access to the 
general curriculum; it is very challenging to monitor whether students are being provided the 
necessary supports to be involved and progress in the general curriculum. This requires looking 
at both special and regular education services. 

We need to maximize our resources even though we are stretched thin at all levels. 
Accountability and partnership at the local level are key. There are not enough people to go 
around, but it can work if there is real ownership at the state and local levels. 

Dialogue about the presentations with all OSEP staff : 

Q: Is OSEP looking at any other strategies for getting parent input given the tendency that 
mostly those with negative input come to public meetings? 

A: The public meetings in each State are structured around focus questions for each 
of the five cluster areas for Part C and the four for Part B. These questions and the 
meeting format are designed to elicit both positive and negative perspectives on special 
education and early intervention services in the State. 

Q: Next spring, state plans will be due. What about the new version? 

A: States are no longer required to submit a “State Plan” to establish Part B eligibility. 
Rather, each State must ensure that it has on file with OSEP documents that meet all 
of the Part B eligibility requirements in section 612(a) of the 1997 amendments to 
IDEA. States are not required to submit documents that OSEP has already approved. 
Because the 1 997 amendments to IDEA requires a number of changes in State policies 
and procedures, States will need to submit many revised documents, together with a 
completed checklist noting those changes, this year by the due date, April 14, 2000. 

Q: Access to the general education curriculum is one part of the new law. What is OSEP 
doing to put the heat on general education to provide services to students with disabilities? 
More heat is still being turned on special education. The public input that special 
education has to endure is not used in other federal program monitoring. 

A: The Department has created a number of crosscutting teams to ensure that other 
programs have explicit and inclusionary statements about students with disabilities. 
OSERS, and the Department as a whole, are working to ensure that responsibility for 
the educational success of children with disabilities is shared between regular 
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education and special education. In its continuous improvement monitoring process, 
OSEP is emphasizing the importance of involvement and progress in the general 
curriculum, and including children with disabilities in State and district-wide 
assessments. Interviews with school administrators and regular education teachers are 
a priority. 

One reality is that IDEA creates an individual entitlement for each child with a 
disability, while the regular education federal programs do not create such an 
individual entitlement for any children. Thus, there is a different kind of leverage 
available at the federal level for special education programs. 

Q: The old monitoring process really did not catch non-compliance unless that child’s 
records were among those reviewed by OSEP. What better ways are being used to satisfy 
all parents who have issues around individual relief? 

A: OSEP continues to use file reviews and interviews to identify systemic compliance 
issues. OSEP will frequently refer individual complaints to the State for it to resolve, 
and OSEP’s monitoring looks closely at the SEA’s and Lead Agency’s complaint 
resolution systems to help ensure that they are effective at identifying and correcting 
both child-specific and systemic noncompliance. 

Q: OSEP’s involvement has been a top-down approach in the past. Is it now a more 
empowering one? If we use the model used in many corporations, it would be more a 
global perspective and more positive. How flexible will OSEP be in working with states 
to develop a monitoring strategy that is empowering and most effective for that particular 
state reflecting the diversity that exists among states? 

A: The most effective way to get the success we want is to use an empowerment 
approach. The most successful of the states in the new monitoring system have been 
those in which the state took ownership. Part of the state’s monitoring system has to 
continue to measure compliance while expanding the capacity to look at results. Any 
state that ignores any part of its compliance responsibilities to work on results could 
not be said to have fulfilled its supervisory responsibilities. 



RRC Luncheon Discussions 



Participants were grouped according to their RRC areas for a luncheon on the first day of the 
conference, and a structured discussion was led by each RRC staff. The topics addressed included 
current monitoring practices and the needs that could be addressed by establishing a “best practices 
in monitoring” site on the internet. The following is an overview of the discussions; the list of 
issues prepared by the RRC staff is contained in Appendix A. 



Current monitoring practices: States are at various stages of movement from straight 
compliance monitoring to a self-assessment/outcome-based/ quality-results oriented approach. One 
of the most prominent features of this change is collaborative planning with input from all 
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concerned stakeholders in the design and implementation of changes. Many states are also actively 
involved in coordinating monitoring for Part B, Part C and 619 as part of their revisions and 
exploring a wide variety of linkages to support the new approach. The common goal of the 
transition is to develop procedures that are both results-based and compliance-based. A variety of 
strategies are being explored for the onsite component, and an LEA core steering committee is 
being added emulating a major element of the OSEP process for states. 

Best practices web site: Participants listed many benefits they could derive from the proposed 
web site including contact information for state monitoring staff, sharing of forms and approaches, 
a Q&A section, and links to useful sites. Only a few concerns were mentioned such as who will 
determine which practices are best (better to use “Promising Practices”), and how the site will be 
maintained. 

Breakout Sessions 

Al. Monitoring Issues for Part C & 619 

Presenters: Tal Black . Associate Director, NECTAS 

Luzanne Pierce . NECTAS at NASDSE 
Cindy Oser. NECTAS at Zero to Three 

This session was designed as an opportunity for Part C and Section 619 staff — those focused on 
the birth to five age group — to meet together to identify and discuss the most challenging aspects 
of monitoring local early intervention and pre-school programs and services. The session did not 
involve formal presentations, but rather was a working session. General discussion and small 
group work resulted in the identification of issues, strategies and promising practices in the 
monitoring of programs and services for this age group. A list of specific items was generated 
by attendees and clustered into the following set of critical topics: 

❖ Transition at age 3 and at age 6 

❖ Multiple agencies/interagency - collaborative monitoring 

❖ Linkages between Parts B & C to foster improved monitoring 

❖ Local ownership including rewards and incentives 

❖ Documentation (e.g., indicators of quality and use of data in monitoring) 

❖ Organizing the process 

❖ Linking technical assistance and monitoring 

❖ Maximizing resources for monitoring 



A2. Statewide Electronic Data Management for Children with Disabilities: An Aid to Monitoring 
Presenters: David Haines . Division of Developmental Disabilities, WY State Department of 

Health 

Mike Witz. The Learning Center Preschool, Jackson, WY 
Connie Kantner . Compliance Coordinator, Bozeman, MT 
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In Wyoming, the challenge of collecting data for children with disabilities birth through five 
was studied by a statewide task force representing the Early Intervention Council, parents, 
providers and various state agencies. The existing system was reviewed in terms of data needs, 
staff expertise and training needs, perceptions, and processes used by other states. As a result, 
Wyoming has adopted a custom tailored electronic reporting system for this population 
including computerized IFSPs and IEPs, management reports used both locally and at the state 
level, and appropriate training for users of the system. Now, staff and parents create 
individualized IEP/IFSP on a laptop and the data is automatically transferred to the state 
management information system. Some of the challenges in designing and implementing this 
new process were involving stakeholders, purchasing and distributing hardware and software, 
training in use of the program, and providing ongoing support for staff. 

A3. Louisiana ’s New Monitoring Process: Focus on Outcomes 

Presenters: Roger James. Division of Special Populations, LA 
Liz Leger. Department of Education, LA 

After almost two years of planning and designing in conjunction with a stakeholders group, the 
new monitoring system for Louisiana was launched in October 1999. The new system is non- 
cyclical, i.e., all data for all systems will be analyzed annually, and visits will be scheduled on 
the basis of performance profiles and not on a cycle. The expressed purpose of the monitoring 
system is: 1) to identify, promote and support best program practices; 2) to ensure program 
effectiveness; and, 3) to enforce legal requirements and measure results of corrective action. 

Using a set of critical indicators, the State Department of Education will develop an annual 
performance profile for each school system based on quantitative data. The indicators include 
exit data, placement in general education, and participation in assessment. The categories and 
type of Part B monitoring they trigger are: 

1) focus = scored the lowest on the critical indicators and will receive on-site visits. 

2) exemplary = scored highest on the critical indicators and will also be visited. 

3) random = systems chosen at random to receive on-site visits. 

4) program assistance = all remaining systems that are not visited that year. Any identified 

areas of need are addressed by other means such as school improvement plans. 

A total of 30% of the LEAs were scheduled for visits during the pilot 1999-2000 school year. 
Qualitative data are reviewed during on-site visits including child find procedures, transition, 
extended school year, discipline, professional development, etc. An initial report is discussed 
with representatives of the system, and a final report is due within 30 days of the completion 
of the visit. The school system then has 20 days to accept or reject the findings and develop a 
plan of corrective action. 

Districts are expected to appoint a local monitoring team to do a self-review. The team will 
monitor regulatory issues for state and federal statutes and summarize their findings in a report 
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that includes best practices identified, non-compliance found, and corrective action. This report 
must be filed with the SEA by July 1 st each year. 

Part C monitoring in Louisiana has been revised and uses the same general approach as 
described above for Part B. Each school/provider will be profiled and designated as focus, 
exemplary, programs assistance, or random. On-site teams for Part C will include professionals 
and parents. The quantitative data profile for Part C includes different data categories such as 
infant/toddler evaluation, family service coordination, and early intervention services. 
Performance indicators have been developed for each category and a new “case validation” is 
being piloted to be used at the program level to assist in improvement efforts. 

Information obtained through the new monitoring process will be used to improve outcomes 
for all children, to direct initiatives statewide, and to direct statewide training activities. 



Bl. The Quality Part of Quality Assurance: Impacting on Program Design and Student Outcomes 
Presenters: Daniel Johnson. Upstate Coordinator, NY 

David Rothfuss , Quality Assurance Regional Associate, NY 
Jane Murphy . Quality Assurance Regional Associate, NY 

At the time of the 1995 National Monitoring Conference, New York staff was seeking help to 
develop a new monitoring system, and used the resources of that conference as well as the 1997 
conference as a significant ingredient in the change process. At this point, New Y ork recognizes 
the importance of the input received from other states and the unique advantage of the peer 
contacts formed through attendance at these national conferences. 

The process began four years ago with the formation of a statewide special education Quality 
Assurance Advisory Group to work with staff of the New York State Education Department's 
Office of Vocational and Educational Services for Individuals with Disabilities (VESID) to 
transform the monitoring system into one of Quality Assurance. The group is composed of 
teachers, parents, school and BOCES (intermediate district) administrators, preschool and private 
school staff, and representatives from the Legislature, New York State School Boards 
Association and other constituencies. 

Conceptually, the system that previously focused upon procedural compliance now focuses on 
program effectiveness and student results. Procedurally, the system which always depended 
upon Department site reviews now involves collaborative program reviews with district staff, 
parents and the Department working together on the development and implementation of plans 
which ensure both procedural compliance and program improvement. Importantly, the goal of 
the collaborative review process is to make programs more effective and improve outcomes for 
students while still ensuring that federal requirements are followed and that the protections 
which are guaranteed to students with disabilities and their parents are enforced. 
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The new system is data driven and results oriented. The review of district-specific data for all 
districts provides the basis for VESID's determination of how often a district will be reviewed 
and the type of review. Whereas, in the past, each district was scheduled for a review once every 
seven years regardless of performance, the Department now establishes its review schedule on 
the basis of demonstrated outcomes for students with disabilities. Thirteen key performance 
indicators that include test scores, diplomas granted, dropout rate, placement and 
disproportionate identification items, are used to categorize districts. VESID staff annually 
review current data related to the 12 key criteria for each district in the State. In addition, staff 
consider information provided through a number of other sources such as districts' IDEA 
Implementation Plans, Regional Space Plans, and parent complaints. These data are used to 
focus staff and other resources by identifying the type of review a district will undergo for the 
following school year: 

♦♦♦ Verification Review Districts - school districts that meet or exceed most or all goals targeted 
by VESID's strategic plan. These districts will not be required to participate in the 
Collaborative Review process, but instead, VESID staff will work with the school districts 
to verify the reported outcome data through a desk audit, confirm that any specific 
compliance issues raised in official complaints brought by parents and others have been 
resolved, provide technical assistance to address deficiencies identified through the desk 
audit, and identify those practices which have helped them achieve these results, for 
dissemination and replication. 

❖ Collaborative Review Districts - districts that exhibit average performance in meeting goals. 

They will participate in the full Collaborative Review process, using the comprehensive set 
of Quality Assurance protocols that have been developed. These districts are expected to 
complete the review process and develop both a Quality Improvement Plan and a 
Compliance Assurance Plan during the school year. 

❖ Facilitated Review Districts - school districts that exhibit a significant discrepancy between 

actual performance and VESID goals in most areas. They will be introduced to the Quality 
Assurance process by spending at least the first year engaged in specific activities designed 
to improve results, according to a plan developed by a team in conjunction with and 
approved by the Department. This plan will include specific technical assistance activities 
to be provided by the Department. After at least one year of technical and financial assis- 
tance, the school district will initiate the Collaborative Review process. 

Collaborative reviews are performed by a team composed of general education and special 
education staff, administrators and parents, as well as the appropriate VESID Regional Associate 
(RA). The team reviews and evaluates the district's performance in key programmatic areas, as 
well as determining procedural compliance. The process also includes formal Surveys of 
students, parents, staff and administrators, both in special education and general education. In 
addition, the Department prepares a district-specific special education data report that provides 
vital information on how students with disabilities are placed, how they achieve and how they 
exit the district's programs. 
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Just as data are used to determine the type, intensity and timing of each review, data developed 
during a collaborative review are used to determine the content of the two plans that mark the 
culmination of each review: 1) the Compliance Assurance Plan reflects procedural violations 
identified during the review and describes the changes in policy or procedure undertaken by the 
district to ensure future compliance; and, 2) the Quality Improvement Plan in which two or more 
priority areas are designated related to achievement and/or integration with non-disabled peers, 
consistent with VESID's Strategic Plan, in which the district seeks to improve its performance. 
In addition to specific action steps, the plan includes timelines, persons responsible and the 
methods by which attainment of the projected outcomes will be evaluated. 

Another major change from the previous monitoring system is the flexible nature of the Quality 
Assurance process and the consistent use of customer feedback to drive system changes. 
Numerous revisions have been made and continue to be made to the original design. Most 
notably, we have moved from a one semester review (either fall or spring) to a year-long process 
as a result of the reports from school districts that additional time was needed to complete the 
review and develop the needed plans for improvement. In response to problems cited by some 
districts in maintaining consistency in review team membership, we now schedule pre-review, 
midway and endpoint meetings with the Superintendent to ensure continuing support for the 
process as the year progresses. 

In addition, based on participant feedback, we have worked with parents and others to 
significantly shorten and simplify the parent survey and to add a student exit survey. We have 
also redesigned the record keeping forms and the format of the Quality Improvement Plan based 
on the participant evaluation team members. A summary of evaluations received from 200 
review team participants in 22 school districts reviewed during the 1997-98 school year, 
including parents, administrators, teachers and other general education and special education 
staff, reflects their strong, positive responses to the new process. 

B2. The Nebraska Quality Improvement Process: Collaboration Toward Quality 

a. Presenters: Barbara Schliesser. Early Intervention Services System, NE 

Joan Luebbers . Early Intervention Services Consultant 
Martha Gadberrv. President, Gadberry & Associates 
Charlie Lewis . Health and Human Services System, NE 
Outcomes? Progress? Results? - The Longitudinal Review Approach 

b. Presenters: Ken Kalamaha. Department of Children, Families and Learning, MN 

Cheryl Staton, Compliance, MN 

a. In April 1 997, the co-lead agencies for the Nebraska Early Intervention Services System 
(Department of Education and the Health and Human Services System) contracted with 
Gadberry and Associates to refine and codify a Quality Improvement Process for the Early 
Intervention Program. The goal of this project was to develop a process that would both measure 
family satisfaction and ensure compliance with regulations. The process that was developed is 
based on a “toolkit” concept to accommodate differences among the regions of the state. For 
example, if a planning region had recently conducted surveys of service providers or families, 
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those tools in the Quality Improvement Review would be redundant. The tools include a self- 
assessment, a file review checklist, and interviews/surveys of providers, lead agencies, families, 
and school district personnel. 

The process was piloted in one region of Nebraska in the fall of 1 997, and revised as a result of 
this work. More than two-thirds of the state have now gone through the Quality Improvement 
Process, and changes are being made based on this review. Forms, interview guides, and other 
information about the monitoring process are included in the Tool Kit. Reports are prepared in 
terms of specific indicators and include documentation of strengths and issues in the areas of 
referral, early identification, assessment, IFSP process, service provision, family rights, and 
services coordination. 

b. In Minnesota, a vital component of the monitoring process is longitudinal review. This review 
addresses quality indicators that help to determine whether or not a student is receiving benefit 
from the special education program and service over time. The longitudinal review addresses the 
following components: 

♦ assessment, evaluation reports, and documentation of eligibility; 

♦ IEP team membership; 

♦ internal consistency of the IEP; and, 

♦ progress across three consecutive IEPs. 

As Minnesota’s monitoring process continues to evolve, we will continue to strive for quality 
and high standards that assist schools across the state to improve their special education systems 
while implementing IDEA. 

B3. Pennsylvania Draft Monitoring Process for Compliance and Continuing Improvement 

a. Presenters: Jill Lichtv . Bureau of Special Education, PA 

Yolanda Yuear. Consultant, Western Instructional Support Center, PA 
Carol Gottron. Director, Western Instructional Support Center, PA 
The West Virginia Continuous Improvement Monitoring Process 

b. Presenters: Kav Johnston. Coordinator, WV 

Jim Parker. Coordinator, WV 
Betsy Scott. Coordinator, WV 
Judy Robertson. Coordinator, WV 

a. The new monitoring process that Pennsylvania is developing is entitled Compliance 
Monitoring for Continuous Improvement (CMCI). Developed with the help of a core workgroup 
from within and outside the state, the CMCI is comprised of a series of components starting with 
the orientation for the district team. The component steps include pre-onsite data collection for 
a facilitated self-assessment including parent and teacher input, followed by an on-site visit to 
verify the data and conduct file reviews. A focused monitoring may then be triggered if data 
from various sources are conflicting or data in any of the four major areas (FAPE in LRE, 
transition, parent involvement and general supervision) fall below designated criteria. 
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Focused monitoring under Pennsylvania’s new system will occur no later than eight weeks after 
the onsite visit. It is designed to be district-specific and may include the gathering of additional 
data, file reviews, classroom observations and student shadowing. After the visit, an 
improvement plan will be developed that includes technical assistance and specific actions, 
responsible personnel and timelines. A three-year review will then be conducted to assess the 
district progress in implementing the improvement plan. 

While continuing to develop and refine the draft plan for the CMCI, a pilot is being planned to 
evaluate the process. Processes and instruments will be revised based on feedback, and a 
procedures manual will be developed upon completion of the pilot stage. 

b. The new West Virginia process is being implemented for the first time in the 1999-2000 
school year. It is designed to focus federal, state and local resources on improved educational 
results for students with disabilities. The cornerstone of the process is a self-assessment 
component which is still in the development stage. 

The West Virginia Continuous Improvement Monitoring Process is a four phase process. Phase 
I is the self-assessment. The local education agency (LEA) will designate a county steering 
committee to assist in the completion of this document. The goal of the self-assessment is to 
provide a data based snapshot of how the county is progressing in areas designated by federal 
indicators and state goals. 

Phase II of the process is the data analysis. The LEA will submit their self-assessment document 
and all applicable documents to the state department. A team at the state level will analyze and 
verify the data submitted. A determination of the extent of the on-site review is determined from 
this analysis. 

Phase III is the on-site review. A team composed of special education (OSE) staff and additional 
personnel from county school districts and Regional Educational Service Agencies will conduct 
the on-site review. Data will be verified through student file review, teacher, parent and 
administrator interviews, a check of administrative practices, and a review of facilities. Within 
60 days of the exit, a report will be issued. 

The fourth phase of the process is the development of an improvement plan. The OSE and LEA 
will meet and collaboratively develop the corrective action plan (CAP). It is expected that this 
improvement plan will become part of the county/school unified improvement plan required by 
the state of West Virginia. 

Cl. Special Education Monitoring in Texas: What ’s New? 

Presenters: Deborah Nance, Senior Director, TX Division of Accountability 

Forrest Now. Director, TX Accountability Development for Special Education 
Pamela Baker, Director, TX Accountability Evaluations for Special Education 
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School districts and campuses in Texas are accountable for meeting the state's standards of 
performance for students and for providing educational programs in accordance with federal and 
state statutes and regulations. On-site evaluations are conducted by trained peer evaluators who 
are guided by professional staff of the Department of Accountability and School Accreditation 
of the Texas Education Agency (TEA). 

One type of accountability evaluation. District Effectiveness and Compliance (DEC), is used for 
conducting district on-site evaluations and monitoring compliance with state and federal 
requirements for special programs, including special education. As a rule, every district will be 
scheduled for a district visit based on a six-year cycle. Additionally, a select number of districts 
are identified for focused special education compliance and effectiveness reviews based upon 
an analysis of risk-based factors correlated to effective educational programming. The on-site 
review process for students served in special education will continue to emphasize the 
importance of compliance with mandated program policies and procedures and, in the 1 999-2000 
school year, will emphasize on the implementation of effective and appropriate special 
educational services. 

A peer evaluation process is established through comprehensive training sponsored annually by 
the Department of Accountability and School Accreditation. Annually, more than one thousand 
local district practitioners, superintendents, principals, program and curriculum specialists, 
counselors, teachers, and other educators are trained as educational program evaluators. They 
subsequently are assigned to participate as peer evaluators, under the guidance of professional 
staff from the Department of Accountability and School Accreditation, in on-site accountability 
evaluations. 

A Reference Guide for Special Education details the Compliance indicators specific to the 
District Effectiveness and Compliance (DEC) System for Special Education. The information 
found in this document is used by the Agency to guide its on-site review of a district’s 
implementation of state and federal mandates for students receiving special education services. 
For each indicator, where appropriate, there are requirements a district would address to be in 
compliance. Sources of information and documentation for indicators have been provided, as 
well as notes the reviewer may use to assist in determining a district’s compliance status. Some 
indicators call for a review of a sample of student folders. To ensure that the sample of folders 
selected during the on-site review is representative of the district’s population of students 
receiving special education services, the Texas Education Agency, with assistance from the 
district, identifies a purposeful sample of students served in the district’s program. Data used 
includes the most recent available to the Agency and the district to ensure sample 
representativeness of the district’s total special education population. During the Agency on-site 
visit, a parent roundtable discussion is held with a representative sample of parents or guardians 
of students receiving special education services. Specific information is gathered regarding the 
experiences of these parents with the district’s special education program. 

The monitoring of special education programs and services for the 1999-2000 school year will 
be incorporating a number of significant changes from previous years. These include: 
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❖ Highlighting the 1997 amendments to IDEA, implementing regulations in the current 

version of the DEC Special Education reference guide; 

❖ Providing on-site follow-up visits to the eleven school districts cited during the 1996 

federal review to verify compliance with required corrective actions; 

❖ Further development and implementation of a focus-based monitoring system to 

complement the current "cyclical-based" system. Based upon an analysis of data 
elements (e.g., dropout rates, statewide assessment of passing rates for special 
education students; placement patterns), districts will be identified for onsite reviews. 
A "risk-based analysis" of district performance will also be used to. focus the 
Agency's monitoring during "cyclical" scheduled visits; 

❖ A Parent Survey has been developed as an additional method to collect relevant 

information about a district's special education programs and services. 

❖ More time will be spent looking at a select number of students during each on-site review. 

Monitors will follow more closely a small number of students, from IEP 
development through LRE placement decision-making to IEP implementation. More 
time will be spent talking to teachers and related service providers about their 
implementation and evaluation of IEP goals for these students. Their responses and 
presentation of specific student-related documentation will assist team members' 
determination of compliance and program effectiveness efforts. 

The sanction framework involves a variety of progressive strategies including: public release of 
compliance review findings, a hearing conducted by the Board of Trustees, submission of 
corrective actions, technical support, follow-up verification visits, assignment of a monitor or 
management team, hearing before the Commissioner, withholding of funds, and lowering of the 
accreditation rating. 

C2. Aligning State Monitoring Systems with OSEP Monitoring Model 
Presenters: Jack Shook. Division Administrator, IL 

Paul Ash . Assistant Director, IN 
Jack Frye-Osier. Consultant, WI 

Pam Williams. Director, Special Education Improvement, MO 

Joyce Jackman. Assistant Director, Special Education Improvement, MO 

In 1997, WI required that the traditional monitoring activities be "refocused' to assist LEAs to 
understand the new requirements in the amended IDEA, assess their compliance, and plan for 
full compliance. The WI special education statute was revised in May 1998, and the old 
administrative rules were repealed in October of that year. The first year that districts must 
implement ALL the changes is 1999-2000, and it is also the last year of refocused monitoring. 
Onsite monitoring activities for the 99-00 school year will again be refocused using self- 
assessment in all LEAs with an emphasis on selected areas where the laws have changed the 
most and are most related to outcomes (overlap with cluster areas). Onsite validation data 
collection (record reviews and interviews) will occur in 96 school districts (half the enrollment). 
In addition, the department will assist all 426 LEAs’ to conduct a self-assessment of their special 
education forms. 
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WI is now working on monitoring for the 2000-2001 school year and beyond. Current planning 
for future monitoring calls for a multiple-element system that includes the LEA Special 
Education Plan, the LEA Special Education Performance Report, and onsite monitoring activities 
that include LEA self-assessments. These elements will align the state’s monitoring system with 
themes and principles in the OSEP monitoring system including: 

1. Continuity (accountability is continuous with self-assessment as a key component), and 

2. Data Driven Process (the continuous improvement process is driven by data that focus on 

improved results). 

The LEA Special Education Plan (SEP) and the LEA Special Education Performance Report 
(SPR) are two newly created provisions in the revised state statute (May 98). These two new 
state statutory provisions, along with the repeal of existing state administrative rules, have 
produced a shift in traditional SEA oversight/monitoring focus and methodology from special 
education program input controls and requirements, input data collection, and primarily SEA 
oversight, to outcome and results data collection, greater public disclosure, and a sharing of the 
oversight function with LEAs, parents, and stakeholders in the community. 

4 

Starting this fiscal year, each LEA in WI must submit a plan for the provision of special 
education to the department. This LEA SEP is a special education program and delivery plan 
document, an evaluation and needs assessment document, a unified special education data 
reporting tool, an IDEA funding application, and a public disclosure mechanism. It creates, for 
the first time, a unified and accessible picture of special education in LEAs. Certain sections 
must be submitted annually and other parts as necessary to reflect data changes or changes to 
LEAs' special education delivery system. Submission of and amendments to the plan via the 
internet are encouraged, with most districts are complying this year. 

One required element is the LEA's plan for evaluating its special education delivery system and 
addressing needs identified by the evaluation. The evaluation must include and be based on, at 
least in part, outcome data that is required to be collected via the SPR, the second new statutory 
provision. 

The SPR requires LEAs to report such data as graduation rates, suspension and expulsion rates, 
and rates of participation in assessments that must include comparison to rates for non-disabled 
children. Other required data include referrals, placements, parent satisfaction and post-school 
outcome information. This data must be reported annually to parents of all children enrolled in 
the LEA and to the SEA. 

The SEP and SPR, standing alone, are monitoring system elements that are data-driven and 
achieve continuous accountability based on a self-assessment system. Accountability is directed 
to the stakeholders and the state. The data can be used as a markers for measuring progress and 
continuous improvement, to support data-based local decision making, and to facilitate oversight 
based on results. These data elements will also be integrated with onsite procedural monitoring. 
When fully integrated beginning in the 2000-2001 school year, it is anticipated that WI’s 
monitoring will incorporate all of the OSEP monitoring themes. 
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WI’s proposed future onsite monitoring activities will more fully develop the OSEP monitoring 
themes. LEAs will be monitored onsite using a six-year cycle. All LEAs will be monitored in 
common cluster areas of the law most related to student outcomes. Additional "customized" self 
assessment and validation data will be collected in each LEA based on data derived from the 
SEP and SPR, viewed in the context of aggregate statewide data and from "benchmarks" from 
the State Improvement Plan process. IDEA complaints, and due process data will also be 
considered. 

Another feature of the new WI monitoring process will occur in the LEAs not scheduled for an 
onsite visit. Annually, all LEAs will evaluate their performance as documented in their SPR and 
these evaluations will be made available to constituents. Consideration is being given to 
requiring that subsequent years’ SEP include activities to address performance areas where the 
LEA significantly trails statewide averages. In addition, all LEAs in the first year of the 
monitoring cycle will be asked to self-assess in the common cluster areas and to make this 
information available to the public. In each of the five remaining years of the cycle, non-onsite 
LEAs will be asked to self assess in areas identified by the SEA based on statewide review of 
LEA SEPs and SPRs. 

On this panel, presenters from Missouri gave a brief introduction to their monitoring system that 
is described in more detail under Session D2 (below). 

C3. Integrating Reform Efforts and Programs: Focused Monitoring in California 
Presenters: Alice Parker . Special Education Division Director, CA 

Alan Coulter . Evaluator, LSU Medical Center 
Mary Hudler, Consultant, CA 
Bernard Yaklin . Research Analyst II, CA 

In describing the development of a new monitoring system, California staff began by reviewing 
the deficiencies of the previous approach. With more than 1,100 districts and over half a million 
students with disabilities, the cyclical monitoring system focused exclusively on compliance, and 
available data were often of little value, frequently inaccurate, and not related to critical issues 
such as student achievement. In planning the new Quality Assurance Process, California 
borrowed from New York where a new monitoring process has been under development for the 
past four years. The components of the new system include comprehensive policies and 
procedures, data on process and results, focused monitoring, effective complaint management, 
and corrective actions and enforcement. 

The new monitoring approach is grounded in the mission of the special education division that 
was developed by a multiple stakeholder group. It includes the statement that “each individual 
participant will achieve in school and post-school outcomes including preparation for the work 
place and independent living.” A broad-based stakeholder group worked with the SEA 
throughout the process. First, key performance indicators (KPI) related to department goals were 
identified and the specific measures for each were selected. It was stressed that the process is not 
driven by data , but rather, the role of data as informing the monitoring process. 
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Using the KPI data, districts are identified as needing different types of focused monitoring: 
a) facilitated reviews - for the lowest performing districts (those with the highest potential for 
program improvement); b) verification reviews - districts are randomly selected from those 
whose documentation reveals general compliance; and, c) preferred practices reviews - high 
performance districts that may be considered for funding in subsequent years to disseminate their 
successes to other districts. 

Currently, the amount of resources needed to implement the new system is being studied. The 
SEA has also contracted with an outside evaluator to provide feedback on the new system and 
the process being used to develop and implement it. The new system is closely integrated with 
the state goals for all children. The State Director described compliance as a floor, not a ceiling. 
Compliance by itself, without high standards, will relegate students with disabilities to the status 
quo or, worse yet, to mediocrity. 

Dl. Vermont ’s Continuous Learning and Improvement Process: Part C of IDEA 

Presenters: Beverly MacCartv, Family, Infant and Toddler Project Director, VT 

Kathy Andrews, 619 Coordinator, VT 
Marisa Edwards, Interagency Coordinating Council, VT 
The Vermont approach for monitoring early intervention programs has five purposes: 1) to 
ensure that families are an integral part of system evaluation; 2) to ensure that early intervention 
services in Vermont meet the needs of infants, toddlers and their families; 3) to promote 
collaboration and communication among state and regional stakeholders; 4) to assist the state 
and regions in identifying strengths, setting goals for improvement, and accomplishing these 
goals; and, 5) to lead to system program improvement. 

A community review is carried out during a site visit every three years. It includes a review of 
processes and procedures, complaints, interagency agreements, training and evaluation activities, 
progress toward goals, and individual child and family records. Volunteer teams are developed 
for each community review and are composed of family members, VT Interagency Coordinating 
Council members, state agency representatives, and community providers. Reports of findings 
include strengths and areas of concern, as well as detailed recommendations and necessary 
actions. These are sent to the region within 45 days of the visit. The region prepares a response 
that contains an action plan developed through the Regional Advisory Team or the local Early 
Childhood Council. It includes input from families, providers, and agencies, and is supposed to 
be specific as to goals and actions. 

The community review process has been found to promote revitalization for regional processes 
of self-reflection and change. It provides a format and framework for planning, implementing, 
and evaluating program and system improvements. 

D2. Show-Me Student Outcomes: Monitoring for Results in Missouri 

Presenters: Pam Williams. Director. Special Education Improvement, MO 

Joyce Jackman. Assistant Director, Special Education Improvement, MO 
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This session presented information about the evolution of Missouri’s special education 
monitoring toward an integrated system with a focus on student outcomes. In 1990 Missouri 
implemented an accreditation process for school districts known as the Missouri School 
Improvement Program (MSIP). MSIP requires an on-site team visit every five years to review 
district programs in three areas - performance, process, and resource. At the time the MSIP was 
implemented, a commitment was made that all SEA monitoring would be completed during the 
MSIP on-site. Although the special education monitoring was conducted in conjunction with 
the MSIP, it remained a separate review with limited integration to the MSIP process. As 
Missouri enters the third cycle of MSIP (2001 -2006), major changes are being proposed for both 
MSIP and special education monitoring. 

In the first two cycles of the MSIP, each of the three areas (performance, process, and resource) 
received equal consideration. In the third cycle — 2001-2006 — student performance will be given 
considerably more weight. Likewise with special education monitoring, in the first cycles, the 
focus was mainly on minimal compliance with the special education process. In the third cycle 
much more emphasis will be placed on documented demonstration of the following: 

□ improvement and maintenance, at a high level, of the performance of students with 

disabilities; 

□ staff awareness and implementation of effective practices and 

□ district support for parent and student participation and collaboration. 

Session participants received information about the proposed changes that will more closely link 
special education services and monitoring with overall school accreditation, how the systems 
will utilize data collection (both aggregated and disaggregated) on various areas of student 
performance, and how waivers may be used for districts that demonstrate high levels of student 
performance. 

D3. Monitoring Through Technology: Self-Assessment and Discipline Reporting 
Presenters: Margaret Lubke. TRL Monitoring, UT 

Debra Cheney. Local Special Education Director, UT 

This presentation began with a discussion of process and outcome monitoring, including an 
historical perspective of the shift to an outcomes focus. Involving stakeholders in the monitoring 
process was addressed in terms of reasons to involve them and ways to make it happen. 
Stakeholders need to be helped to assess their knowledge of the special education process and 
to realize that there are useful things to be learned. 

Statistical data analysis, especially hierarchical linear modeling, allows accurate analysis of data 
and a variety of valuable measures. Technology can be used to monitor discipline requirements. 
The Research Institute for Safe and Effective Programs (RISEP) has been funded through the 
Safe and Drug Free Schools and Communities program of the U.S. Department of Education to 
develop an information collection and dissemination center established to help school 
administrators and personnel monitor and prevent substance abuse and violence. The RISEP 
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Incident System includes a suspension log and an incident report form. It can be viewed on the 
internet at: www.risep.usu.edu. 

This session also included discussion of minimum compliance documentation for functional 
behavioral assessment and for behavioral intervention plans, as well as documenting 
manifestation determinations. 



Closing Plenary: State Innovative Practices 

Presenters: Robert Pasternak, State Director of Special Education, NM 
Cathy Stevenson . Part C Director, NM 
Peggy Blanton. Manager, KY 
Barbara Kibler, Attorney, KY 
Gregg Corn Associate Division Director, OSEP 

A unique partnership exists between the Part C and Part B programs in New Mexico. This 
collaboration includes an important ongoing relationship with federal partners at OSEP. The 
presenters included discussion of the federal and state perspective from Part C and Part B, and 
offered strategies for replicating this unique partnership in other states. Highlights from the 
presentation follow: 

New Mexico sees many advantages to collaboration around the OSEP State Monitoring Process, 
expecting that the partnering process will help assure that the IDEA System, both Part B and Part 
C, is meeting the needs of stakeholders. The federal visit is seen as a way to evaluate and 
improve state efforts on behalf of children and families. The information gathered and the joint 
efforts are a foundation and ongoing support for program growth within both agencies. In order 
to maximize the usefulness of this effort. New Mexico has attempted to coordinate measurement 
strategies and management of some common elements at the state level. Work has continued in 
the months since the OSEP visit to meet requirements in a proactive and positive manner. 
Examples of collaborative systems initiatives currently under way include: 

• Joint annual meetings on of the IDEA Panel and the ICC ( Executive Committees will 
meet twice each year) 

• A joint committee on transition between the two systems was formed during the OSEP 
process and continues to meet. Work of the committee includes "crosswalking" NM Part 
B, and Part C Regulations and developing a broad multi-agency transition agreement. 

• A task force to develop and oversee the pilot process for a common IFSP-IEP form for 
children from birth to age nine. A preliminary form has been drafted and timelines for 
carrying out the remainder of the project are in place. 

• Work on the NM CSPD, led by Part B 619 staff, includes exciting new initiatives focused 
on the development of paraeducators at both the in-service and pre-service levels. 

• New Mexico uses a common pool of trained hearing officers and mediators for the Part B 
and Part C systems. Last year a Dispute Resolution Task Force was formed to look at ways 
of reducing complaints within the system as well as ways to more effectively deal with 
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complaints received. As a result of this work the New Mexico Part C system is supporting 
the statewide training on Collaborative Negotiation Skills for parents, providers and public 
school personnel. 

Tips for collaboration and general guidelines based on what has worked in New Mexico include: 

• Know your own program and be sure about what you can commit to, compromise on and 
live with. 

• Learn all you can about your partners' programs and have realistic expectations. 

• Establish a common vision for your work together. 

• Engage stakeholders and stakedrivers. 

• Allow extra time for "leg work" on all sides. 

• Honor your commitments. 

• Use humor when appropriate. 

• Celebrate the small steps as well as the major victories. 

The NM session concluded with comments from the presenters about the dilemmas facing 
education systems, students, families and children today and the urgency of working together 
as well as individually to address these issues. 

Kentucky also participated in the closing plenary session providing information on a special 
component of their system, school-specific monitoring. Each year districts may volunteer, and 
a few are chosen by the SEA, for a school-by-school monitoring process and report. The school 
report is specific to areas of concern in that school in contrast to the district report that combines 
information from all schools into one report. 

Directors of special education have asked for school specific monitoring because they think their 
district has had district wide professional development and they are now ready to fine tune the 
skills of the staff in each specific school. Sometimes they feel the majority of their schools are 
doing well, but have one or two that are not making the kind of progress toward compliance they 
would like to see and therefore can benefit from a school specific report. 

The Kentucky SEA may direct a school specific monitoring when there are indicators that the 
district is in trouble. If the school district has had several complaints or due process hearings or 
if they have been unable to fulfill the requirements of a corrective action plan, KDE may choose 
to conduct a school specific monitoring visit. The benefits to the schools that have already been 
monitored in this fashion are increased awareness of procedures of the school administration and 
faculty, targeted corrective action to the specific issues or personnel, and ownership of the 
corrective action plan by the entire school. 

CONFERENCE SUMMARY AND NEXT STEPS 

Although the major focus of the 1999 National Conference was state monitoring of local 
districts, OSEP staff made a formal presentation and participated on an informal basis throughout 
the Conference. The OSEP presentation provided a review of the status of the Continuous 
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Improvement Monitoring Process, their new approach to monitoring states. They summarized the 
structure of the new results-based system and described their experience after the first full year of 
implementation. 

State-level presentations at this conference portrayed state monitoring systems undergoing 
significant systemic change. Every state described plans to align their monitoring procedures with 
the new OSEP approach, with some adding their own unique versions of the continuous 
improvement elements. In each case, the change effort started with a broad-based stakeholder 
working group/committee that helped to design, plan, and implement the new system. The adoption 
or enhancement of a district self-assessment is another common element of the revisions. 

The collection and use of data as the basis of monitoring is one of the more challenging aspects 
of the new approach. States are at different levels in their development of capacity in this area, 
especially with regard to aggregating and disaggregating data. The availability of special education 
data as well as the ability to make comparisons to general education data is only beginning to 
develop in some states. 

For the first time, conference participants from Part C and pre-school programs were able to 
share information on the similarities and differences in their monitoring activities, providing a basis 
for further dialogue on this topic at the state level. Their participation stimulated extensive 
discussions of how to coordinate monitoring across the full special education age span, while 
accommodating the unique elements of each of the programs. 

States recognize their continuing obligation to verify district implementation of all requirements, 
but they expressed concerns about the unproductive process of cyclical, routine onsite visits that 
have characterized past monitoring systems. Many are developing elements of focused monitoring 
that will concentrate on student performance outcomes and the successful resolution of past or 
present non-compliance issues. In addition, states are recognizing exemplary practices during the 
monitoring process, with some adding rewards to such commendations. State monitoring staff 
expressed a commitment to the basic philosophy of a partnership approach with local districts, but 
they continue to express concerns about the appropriate balance between the sometimes competing 
tasks of monitoring for compliance and providing technical assistance. Most states are now piloting 
their revised approaches. The issues they are continuing to address include appropriate involvement 
of families in monitoring, and meeting the expectations of a new system in the face of reduction of 
state agency staffing levels. 

Readers familiar with the proceedings documents for the prior National Monitoring conferences 
will realize that this document does not include data from a survey on monitoring practices in all 
states. The workgroup decided that conducting a survey in 1 999 would not be appropriate since most 
states are in the process of revising their monitoring systems. It was decided that a state survey will 
be carried out after the 1 999-2000 school year, and the results will be made available to all interested 
parties through a variety of internet sites including those of OSEP and the FRRC network. 
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As mentioned above in the summary of the first day luncheon discussions, one other activity is 
being explored by the Monitoring Workgroup as a follow-up to this conference — the development 
of a website for successful practices in state monitoring of local districts. Participants expressed 
interest in having an ongoing opportunity to share ideas and practices, and the website will be one 
strategy for addressing that need. 
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NATIONAL MONITORING CONFERENCE 
SEA/ RRC LUNCHEONS 
OCTOBER 19, 1999 



ISSUES: 

MONITORING PRACTICES 

1. Where is each state in terms of current monitoring practices and policies? 

Self-improvement focus at district level 
Part C self-survey 

Collaboration between Part C and Head Start 

Proactively doing self-assessment and focusing on systemic issues 

Focused follow-up 

Family satisfaction survey on issues such as transition, compliance and quality 
Data drawn to school levels on standards 

Compliance data collected for OCR bi-lingual education programs and state reform in addition to 
IDEA 

Heavily into data and compliance driven systems 
Disaggregation of data by district to create district profiles 
Collection and review of many sources of data 

On-site monitoring that generally takes one week, with on-site follow-up of CAP the following 
year 

Separate monitoring efforts with Part B and C 

Movement away from straight compliance monitoring to self-assessment/outcome 
-based/quality/results oriented models 

Continuing to do Compliance monitoring but plan to expand collaborative planning especially 
between parents and district level personnel 
New quality assurance for Part C, organized around five major outcomes as its focus 
Self-assessment tool and 1 999 procedural manual for LEAs 
LEA monitoring includes calls to local parents as part of data collection 
Alignment of SIG activities/stakeholder focus with evolving monitoring system 
Empowerment of LEAs to guide their own self-assessment process 
Student services coordinators being trained as part of SIG 
Web site for special education data 

Quality Assurance Process developed, consisting of a data informed process around five broad 
goals. QAP is to be aligned with all other educational approaches 
New consequences for LEAs being developed, in addition to working with LEAs to develop new 
partnerships and new expectations to achieve new and better outcomes 
Parents and especially the PTI are included in the process of change 

One year (voluntary) self-assessment monitoring model, which is research oriented with LEA 
teams gathering data, developing focus question, gathering evidence and designing a plan to 
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address the needs identified and documented. The process includes case studies and 
stakeholder interviews. There is funding and on-site technical assistance providers to 
implement action plans. The monitoring function is contracted out to a vendor. Although the 
process is currently voluntary, the SEA is considering utilizing this process for LEAs that are 
grossly out of compliance. The SEA is moving to include preschool programs within the 
self-assessment model and instituting on-site observation of Part H programs as well 
New comprehensive monitoring system has been instituted which has two parts - compliance 
and an additional program results piece. It incorporates self-assessment with compliance and 
results. The SEA will be monitoring 100 LEAs /year and will be initiating stakeholder 
interviews and focus groups (including parents) as part of its monitoring process of the LEAs 
Monitoring procedures are both results and compliance based. 12 key indicators drive the review 
process, with three different levels of review - the level of review depends on the district data 
collected. Monitoring policies and procedures are highly collaborative. LEA monitoring 
teams are stakeholder based-teams that include parents, principals, etc. Team members 
complete a participant evaluation form and surveys are used for other stakeholders including 
students. The SEA also collects exit data. The result of the process has been improved 
results and compliance 

Three-day on-site monitoring visit, with first day for record review, second day interviews with 
staff (based on record review) and third day interviews with administrators. Technical 
assistance is provided on-site in any area of non-compliance. All services providers, 
paraprofessionals and parents - general and special education - are using data notebooks. A 
comprehensive system of supports exists in each school, including educational support teams 
Working with 1 5 person core steering committee to develop the Improvement Plan 
Waiting for final OSEP report to develop Corrective Action and Improvement Plans 
Identified four stakeholder groups to address concerns expressed by OSEP in exit meeting. 

Waiting for final OSEP report to finalize its Improvement Plans. Task force groups will have 
results and final recommendations by next spring 
Holding planning meetings and working with Steering Committee prior to monitoring 
Working with combined ICC, Advisory Committee and Steering Committee to address concerns 
expressed in OSEP exit interviews 

Working on State Plan and moving ahead with new monitoring system 
Completing self-assessment prior to OSEP on-site visit 

In process of developing self-assessment and working with newly formed Steering Committee 
and combining Part B and C 

2. What (monitoring) linkages are there currently between Parts B and C in your state? 

Collaboration of core teams - Education/ Health and Human Services /Head Start/ Department of 
Health/ Other Part C lead agencies 
Sub-committee of SIP and monitoring Part B through age 21 
Interagency agreements - monitoring at the district level 
Effective transition team for Part C and 619 
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With Parts B and C located with the State Department of Education, Part C and 619 are an 
integrated seamless system, with Part C monitoring integrated into overall special education 
monitoring 

Participation of 619 and Part G in each other’s monitoring visits, including on-site visits 
Development of performance goals in conjunction with other stakeholders for birth through 21 
Looking at closer collaboration between Parts B and C 

Strong collaboration between Parts B and C - HHS, which is the lead agency, moved to a self- 
assessment process monitoring model similar to the one used by Part B. Pilot project just 
completed 
No collaboration 

Collaboration in monitoring depends upon whether the Part C program is approved by the 
Department of Education or the Part C lead agency - all reports are shared across the two 
entities 

Department of Health collects data for child count 
Revision of Part C and collaborating with Part B 
Combining Parts B and C 
Core team of Part C / Head Start/ Health/ etc. 

Subcommittee of SIP and monitoring birth - 2 1 
Interagency agreements - monitor at district level 

3. Other Part B and C issues 

Focus on interagency issues - entitlement versus mandate and how to monitor interagency and 
respite 

BEST PRACTICES WEB SITE 

1. What needs could best be addressed with web site showcasing state level practices? 

Contact information for each state - list of names, addresses, telephone numbers and web sites 
Sharing of forms and documents that can be downloaded 
Four targeted OSEP cluster areas 

Practices that are working and not working in other states 

Q&A section for directors to ask questions and post ideas regarding best practices 
Chat room (Suggestion that it be closed to explore and discuss best practice) 

List serve 

List of states already monitored and OSEP monitoring reports from Parts B and C 
Concerns surrounding implementation of continuous improvement 
TA & D network 

Data (examples: what data supports best practices and student results both in and out of school; 
how data is collected and gathered) 

Demographics regarding other state issues, including links to other states’ web sites 
Links to parents /families 
List of consultants 
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Links to states on specific data regarding alternate assessment, data profiles, monitoring 
procedures and protocols 

Links to all SEA home pages, RRCs, OSEP, NECTAS, etc. 

OSEP’s performance indicators - examples and processes from different states 
Posting of meetings and training events - national and regional 
Post-secondary tracking models 
Include general education 

State tracking of deficiencies and corrective action 
Consequences (corrective action) that make a difference 
Organization plans 

Resources, i.e., graduation requirements, guidelines, discipline reporting, etc. 

Funding projects, sources of grants, model programs 

SIGs and SIPs - integrating with monitoring system / Part B and C collaborating on 
0 How monitoring and state accountability systems integrate 
0 Using monitoring mechanisms established by other programs 
0 Best practices specific to rural/remote areas 

0 Data profiles requested from districts and used by SEA for monitoring purposes 
0 Research 

0 Web Map with quick access 
0 Evaluation/ results information 

0 Links to related general education documents in curriculum, instruction, etc. 

0 IEP formats/ access to general curriculum that could be downloaded 
0 Copy of monitoring manuals with name of contact person 
0 Accountability monitoring documents with links to other sites 
0 FAQs 

0 Posting of ideas from State Directors regarding best practice in their states 
0 State strategies around partnerships 
0 Workshops on IEP composition and transition 
0 Gauging and evaluating parent satisfaction 
0 Birth mandate - Section 619 versus Part C 
0 Cross referencing of state resources : LRP 
0 Evaluation review process in manifestation determinations 
0 Identification of natural environment 
0 Appropriate activities - top/down 
0 Interpretation of access (other states) 

2. What information in what formats would be most helpful? 

As many links as possible, especially hot links to state’s home page 

Designated and organized according to the subject matter 

Minimize graphic so as not to slow down access 

Ability to click back to prior link 

Chat room /List serve 

Searchable site 
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Easy to remember URL 
Variety of formats (Acrobat) 

Link to conference web sites 
Email addresses/contact information 
Opportunity to solicit information 
Interactive web site 
Highlighted on front page 

Use format similar to Minnesota’s web site - others suggested were NH and NY 
Glossary of terminology 

“Quick look” list with name of contact (fast-read format such as bullets instead of text) 

Access by category and by state 

Information from OSEP regarding who is doing this well 
Promising practices rather than best practices 

Organize into separate Part B and C sections, then combined collaborations issues 
Eligibility criteria for Part C 

3. Miscellaneous 

Concern over title (Who will determine which practices are best?) Suggest “Recommended 
Practices” or “Promising Practices” 

Research the benefits first, i.e., what has/has not worked 

Suggest contacting Consortium for School Linked Services (California) 

No interest in Best Practices in Monitoring web site; however, would like to be kept apprized of 
activities being planned and implemented 
Will all states have access to the web site? 

The data base will need to be updated regularly 

What kind of information would be on the web site and who would decide what information is 
included 

What would be the scope? There would need to be room for expansion 
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